{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zBQDGVIe95Sj"
   },
   "source": [
    "# EasyEdit Example with **IKE**\n",
    "> Tutorial author: Ting Lu(<luting1@shanghaitech.edu.cn>) and Bozhong Tian(<tbozhong@zju.edu.cn>)\n",
    "> \n",
    "In this tutorial, we use `IKE` to edit `gpt2` ans `LlaMA` model. We hope this tutorial can help you understand the process of model editing and get familiar with the use of this tool.\n",
    "\n",
    "This tutorial uses `Python3.9`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cbn0z6or-FPa"
   },
   "source": [
    "# Model Editing\n",
    "![Model Editing to fix and update LLMs]()\n",
    "\n",
    "Deployed models may still make unpredictable errors. For example, Large Language Models (LLMs) notoriously *hallucinate*, *perpetuate bias*, and *factually decay*, so we should be able to adjust specific behaviors of pre-trained models.\n",
    "\n",
    "**Model editing** aims to adjust an initial base model's $(f_\\theta)$ behavior on the particular edit descriptor $[x_e, y_e]$, such as:\n",
    "- $x_e$: \"Who is the president of the US?\n",
    "- $y_e$: \"Joe Biden.\"\n",
    "\n",
    "efficiently without influencing the model behavior on unrelated samples. The ultimate goal is to create an edited model$(f_\\theta’)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7WQuYKbU_jDn"
   },
   "source": [
    "# Editing Scope\n",
    "![scope.png]()\n",
    "\n",
    "The model editing process generally impacts the predictions for a broad set of inputs **that are closely** associated with the edit example, called the **editing scope**.\n",
    "\n",
    "\n",
    "A successful edit should adjust the model’s behavior within the editing scope while remaining unrelated inputs(as below formula).\n",
    "\n",
    "\n",
    "$f_{\\theta_{e}}(x) = \\begin{cases}\n",
    "y_e & \\text{if } x \\in I(x_e,y_e) \\\\\n",
    "f_{\\theta}(x) & \\text{if } x \\in O(x_e, y_e) \\end{cases}$\n",
    "\n",
    "In addition to this, the performance of model editing should be measured from multiple dimensions:\n",
    "\n",
    "- `Reliability`: the success rate of editing with a given editing description\n",
    "- `Generalization`: the success rate of editing **within** the editing scope\n",
    "- `Locality`: whether the model's output changes after editing for unrelated inputs\n",
    "- `Portability`: the success rate of editing for factual reasoning(one hop, synonym, one-to-one relation)\n",
    "- `Efficiency`: time and memory consumption required during the editing process\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yx9VP0adjNmf"
   },
   "source": [
    "# Method: **IKE**\n",
    "\n",
    "Paper: [Can We Edit Factual Knowledge by In-Context Learning?](https://arxiv.org/abs/2305.12740)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KBphuoMBh7Ec"
   },
   "source": [
    "**IKE** (In-context Knowledge Editing), is a way of editing factual knowledge in large language models **without modifying their parameters**, but by **providing different types of natural language demonstrations** as part of the input.  \n",
    "It can achieve competitive knowledge editing performance **with less computation overhead and side effects**, as well as better scalability and interpretability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z0mWT8G9h25Q"
   },
   "source": [
    "![image.png]()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MiEzUSIak_eu"
   },
   "source": [
    "## Prepare the runtime environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RO20sOmEqq-O",
    "outputId": "5512ce5d-cc32-4420-9bda-f5310cf4c531"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/mnt/8t/xkw/EasyEdit\n",
      "colab_requirements.txt\tedit.py   LICENSE\t      requirements.txt\n",
      "demo\t\t\texamples  logs\t\t      results\n",
      "Dockerfile\t\tfigs\t  multimodal_edit.py  tutorial-notebooks\n",
      "easyeditor\t\thparams   README.md\t      tutorial.pdf\n"
     ]
    }
   ],
   "source": [
    "## Clone Repo\n",
    "!git clone https://github.com/zjunlp/EasyEdit\n",
    "%cd EasyEdit\n",
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "w1eq_DOx9_2e",
    "outputId": "0bd4fe69-8eb3-4bc0-ed90-6ae67b6fe7a1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading package lists... Done\n",
      "Building dependency tree       \n",
      "Reading state information... Done\n",
      "The following additional packages will be installed:\n",
      "  libpython3.9-minimal libpython3.9-stdlib python3.9-minimal\n",
      "Suggested packages:\n",
      "  python3.9-venv binfmt-support\n",
      "The following NEW packages will be installed:\n",
      "  libpython3.9-minimal libpython3.9-stdlib python3.9 python3.9-minimal\n",
      "0 upgraded, 4 newly installed, 0 to remove and 15 not upgraded.\n",
      "Need to get 5,030 kB of archives.\n",
      "After this operation, 19.7 MB of additional disk space will be used.\n",
      "Get:1 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 libpython3.9-minimal amd64 3.9.17-1+focal1 [805 kB]\n",
      "Get:2 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 python3.9-minimal amd64 3.9.17-1+focal1 [2,064 kB]\n",
      "Get:3 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 libpython3.9-stdlib amd64 3.9.17-1+focal1 [1,666 kB]\n",
      "Get:4 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 python3.9 amd64 3.9.17-1+focal1 [495 kB]\n",
      "Fetched 5,030 kB in 4s (1,145 kB/s)\n",
      "Selecting previously unselected package libpython3.9-minimal:amd64.\n",
      "(Reading database ... 123105 files and directories currently installed.)\n",
      "Preparing to unpack .../libpython3.9-minimal_3.9.17-1+focal1_amd64.deb ...\n",
      "Unpacking libpython3.9-minimal:amd64 (3.9.17-1+focal1) ...\n",
      "Selecting previously unselected package python3.9-minimal.\n",
      "Preparing to unpack .../python3.9-minimal_3.9.17-1+focal1_amd64.deb ...\n",
      "Unpacking python3.9-minimal (3.9.17-1+focal1) ...\n",
      "Selecting previously unselected package libpython3.9-stdlib:amd64.\n",
      "Preparing to unpack .../libpython3.9-stdlib_3.9.17-1+focal1_amd64.deb ...\n",
      "Unpacking libpython3.9-stdlib:amd64 (3.9.17-1+focal1) ...\n",
      "Selecting previously unselected package python3.9.\n",
      "Preparing to unpack .../python3.9_3.9.17-1+focal1_amd64.deb ...\n",
      "Unpacking python3.9 (3.9.17-1+focal1) ...\n",
      "Setting up libpython3.9-minimal:amd64 (3.9.17-1+focal1) ...\n",
      "Setting up python3.9-minimal (3.9.17-1+focal1) ...\n",
      "Setting up libpython3.9-stdlib:amd64 (3.9.17-1+focal1) ...\n",
      "Setting up python3.9 (3.9.17-1+focal1) ...\n",
      "Processing triggers for man-db (2.9.1-1) ...\n",
      "Processing triggers for mime-support (3.64ubuntu1) ...\n"
     ]
    }
   ],
   "source": [
    "!apt-get install python3.9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zFakniN_OxvU",
    "outputId": "498a305d-2d39-411a-b947-3218a3872ed0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 3 choices for the alternative python3 (providing /usr/bin/python3).\n",
      "\n",
      "  Selection    Path                 Priority   Status\n",
      "------------------------------------------------------------\n",
      "* 0            /usr/bin/python3.10   2         auto mode\n",
      "  1            /usr/bin/python3.10   2         manual mode\n",
      "  2            /usr/bin/python3.8    1         manual mode\n",
      "  3            /usr/bin/python3.9    1         manual mode\n",
      "\n",
      "Press <enter> to keep the current choice[*], or type selection number: 3\n",
      "update-alternatives: using /usr/bin/python3.9 to provide /usr/bin/python3 (python3) in manual mode\n"
     ]
    }
   ],
   "source": [
    "!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1\n",
    "!sudo update-alternatives --config python3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "V7i-2uXDYXAN",
    "outputId": "07f4c882-56a9-4b07-a88d-bde275344a15"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading package lists... Done\n",
      "Building dependency tree       \n",
      "Reading state information... Done\n",
      "The following additional packages will be installed:\n",
      "  python-pip-whl python3-setuptools python3-wheel\n",
      "Suggested packages:\n",
      "  python-setuptools-doc\n",
      "The following NEW packages will be installed:\n",
      "  python-pip-whl python3-pip python3-setuptools python3-wheel\n",
      "0 upgraded, 4 newly installed, 0 to remove and 15 not upgraded.\n",
      "Need to get 2,389 kB of archives.\n",
      "After this operation, 4,933 kB of additional disk space will be used.\n",
      "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python-pip-whl all 20.0.2-5ubuntu1.9 [1,805 kB]\n",
      "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-setuptools all 45.2.0-1ubuntu0.1 [330 kB]\n",
      "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-wheel all 0.34.2-1ubuntu0.1 [23.9 kB]\n",
      "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-pip all 20.0.2-5ubuntu1.9 [231 kB]\n",
      "Fetched 2,389 kB in 1s (1,644 kB/s)\n",
      "Selecting previously unselected package python-pip-whl.\n",
      "(Reading database ... 123743 files and directories currently installed.)\n",
      "Preparing to unpack .../python-pip-whl_20.0.2-5ubuntu1.9_all.deb ...\n",
      "Unpacking python-pip-whl (20.0.2-5ubuntu1.9) ...\n",
      "Selecting previously unselected package python3-setuptools.\n",
      "Preparing to unpack .../python3-setuptools_45.2.0-1ubuntu0.1_all.deb ...\n",
      "Unpacking python3-setuptools (45.2.0-1ubuntu0.1) ...\n",
      "Selecting previously unselected package python3-wheel.\n",
      "Preparing to unpack .../python3-wheel_0.34.2-1ubuntu0.1_all.deb ...\n",
      "Unpacking python3-wheel (0.34.2-1ubuntu0.1) ...\n",
      "Selecting previously unselected package python3-pip.\n",
      "Preparing to unpack .../python3-pip_20.0.2-5ubuntu1.9_all.deb ...\n",
      "Unpacking python3-pip (20.0.2-5ubuntu1.9) ...\n",
      "Setting up python3-setuptools (45.2.0-1ubuntu0.1) ...\n",
      "Setting up python3-wheel (0.34.2-1ubuntu0.1) ...\n",
      "Setting up python-pip-whl (20.0.2-5ubuntu1.9) ...\n",
      "Setting up python3-pip (20.0.2-5ubuntu1.9) ...\n",
      "Processing triggers for man-db (2.9.1-1) ...\n",
      "Collecting aiohttp==3.8.1\n",
      "  Downloading aiohttp-3.8.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.2 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.2 MB 6.1 MB/s \n",
      "\u001b[?25hCollecting aiosignal==1.2.0\n",
      "  Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB)\n",
      "Collecting antlr4-python3-runtime==4.8\n",
      "  Downloading antlr4-python3-runtime-4.8.tar.gz (112 kB)\n",
      "\u001b[K     |████████████████████████████████| 112 kB 71.5 MB/s \n",
      "\u001b[?25hCollecting anyio==3.6.2\n",
      "  Downloading anyio-3.6.2-py3-none-any.whl (80 kB)\n",
      "\u001b[K     |████████████████████████████████| 80 kB 11.6 MB/s \n",
      "\u001b[?25hCollecting argon2-cffi==21.3.0\n",
      "  Downloading argon2_cffi-21.3.0-py3-none-any.whl (14 kB)\n",
      "Collecting argon2-cffi-bindings==21.2.0\n",
      "  Downloading argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)\n",
      "\u001b[K     |████████████████████████████████| 86 kB 6.4 MB/s \n",
      "\u001b[?25hCollecting arrow==1.2.3\n",
      "  Downloading arrow-1.2.3-py3-none-any.whl (66 kB)\n",
      "\u001b[K     |████████████████████████████████| 66 kB 5.6 MB/s \n",
      "\u001b[?25hCollecting asttokens==2.2.1\n",
      "  Downloading asttokens-2.2.1-py2.py3-none-any.whl (26 kB)\n",
      "Collecting async-timeout==4.0.2\n",
      "  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n",
      "Collecting attrs==22.2.0\n",
      "  Downloading attrs-22.2.0-py3-none-any.whl (60 kB)\n",
      "\u001b[K     |████████████████████████████████| 60 kB 8.5 MB/s \n",
      "\u001b[?25hCollecting backcall==0.2.0\n",
      "  Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)\n",
      "Collecting backports.csv==1.0.7\n",
      "  Downloading backports.csv-1.0.7-py2.py3-none-any.whl (12 kB)\n",
      "Collecting base58==2.1.1\n",
      "  Downloading base58-2.1.1-py3-none-any.whl (5.6 kB)\n",
      "Collecting beautifulsoup4==4.12.2\n",
      "  Downloading beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)\n",
      "\u001b[K     |████████████████████████████████| 142 kB 82.1 MB/s \n",
      "\u001b[?25hCollecting black==22.1.0\n",
      "  Downloading black-22.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.5 MB 52.5 MB/s \n",
      "\u001b[?25hCollecting bleach==6.0.0\n",
      "  Downloading bleach-6.0.0-py3-none-any.whl (162 kB)\n",
      "\u001b[K     |████████████████████████████████| 162 kB 80.1 MB/s \n",
      "\u001b[?25hCollecting blessed==1.20.0\n",
      "  Downloading blessed-1.20.0-py2.py3-none-any.whl (58 kB)\n",
      "\u001b[K     |████████████████████████████████| 58 kB 6.3 MB/s \n",
      "\u001b[?25hCollecting blis==0.7.5\n",
      "  Downloading blis-0.7.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.9 MB)\n",
      "\u001b[K     |████████████████████████████████| 9.9 MB 111 kB/s \n",
      "\u001b[?25hCollecting boto3==1.20.47\n",
      "  Downloading boto3-1.20.47-py3-none-any.whl (131 kB)\n",
      "\u001b[K     |████████████████████████████████| 131 kB 81.8 MB/s \n",
      "\u001b[?25hCollecting botocore==1.23.47\n",
      "  Downloading botocore-1.23.47-py3-none-any.whl (8.5 MB)\n",
      "\u001b[K     |████████████████████████████████| 8.5 MB 75.7 MB/s \n",
      "\u001b[?25hCollecting cachetools==5.0.0\n",
      "  Downloading cachetools-5.0.0-py3-none-any.whl (9.1 kB)\n",
      "Collecting catalogue==2.0.6\n",
      "  Downloading catalogue-2.0.6-py3-none-any.whl (17 kB)\n",
      "Collecting certifi==2022.12.7\n",
      "  Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)\n",
      "\u001b[K     |████████████████████████████████| 155 kB 65.8 MB/s \n",
      "\u001b[?25hCollecting cffi==1.15.1\n",
      "  Downloading cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)\n",
      "\u001b[K     |████████████████████████████████| 441 kB 71.5 MB/s \n",
      "\u001b[?25hCollecting chardet==4.0.0\n",
      "  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)\n",
      "\u001b[K     |████████████████████████████████| 178 kB 79.7 MB/s \n",
      "\u001b[?25hCollecting charset-normalizer==2.0.11\n",
      "  Downloading charset_normalizer-2.0.11-py3-none-any.whl (39 kB)\n",
      "Collecting checklist==0.0.11\n",
      "  Downloading checklist-0.0.11.tar.gz (12.1 MB)\n",
      "\u001b[K     |████████████████████████████████| 12.1 MB 16.0 MB/s \n",
      "\u001b[?25hCollecting cheroot==8.6.0\n",
      "  Downloading cheroot-8.6.0-py2.py3-none-any.whl (104 kB)\n",
      "\u001b[K     |████████████████████████████████| 104 kB 63.9 MB/s \n",
      "\u001b[?25hCollecting CherryPy==18.6.1\n",
      "  Downloading CherryPy-18.6.1-py2.py3-none-any.whl (419 kB)\n",
      "\u001b[K     |████████████████████████████████| 419 kB 67.1 MB/s \n",
      "\u001b[?25hCollecting click==8.0.3\n",
      "  Downloading click-8.0.3-py3-none-any.whl (97 kB)\n",
      "\u001b[K     |████████████████████████████████| 97 kB 8.3 MB/s \n",
      "\u001b[?25hRequirement already satisfied: comm==0.1.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 31)) (0.1.3)\n",
      "Collecting commonmark==0.9.1\n",
      "  Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)\n",
      "\u001b[K     |████████████████████████████████| 51 kB 8.0 MB/s \n",
      "\u001b[?25hCollecting cryptography==36.0.1\n",
      "  Downloading cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)\n",
      "\u001b[K     |████████████████████████████████| 3.8 MB 62.5 MB/s \n",
      "\u001b[?25hCollecting cycler==0.11.0\n",
      "  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)\n",
      "Collecting cymem==2.0.6\n",
      "  Downloading cymem-2.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (35 kB)\n",
      "Collecting datasets==1.18.3\n",
      "  Downloading datasets-1.18.3-py3-none-any.whl (311 kB)\n",
      "\u001b[K     |████████████████████████████████| 311 kB 78.3 MB/s \n",
      "\u001b[?25hRequirement already satisfied: debugpy==1.6.7 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 37)) (1.6.7)\n",
      "Requirement already satisfied: decorator==5.1.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 38)) (5.1.1)\n",
      "Requirement already satisfied: defusedxml==0.7.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 39)) (0.7.1)\n",
      "Collecting dill==0.3.4\n",
      "  Downloading dill-0.3.4-py2.py3-none-any.whl (86 kB)\n",
      "\u001b[K     |████████████████████████████████| 86 kB 6.0 MB/s \n",
      "\u001b[?25hCollecting docker-pycreds==0.4.0\n",
      "  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n",
      "Collecting einops==0.4.0\n",
      "  Downloading einops-0.4.0-py3-none-any.whl (28 kB)\n",
      "Collecting entrypoints==0.4\n",
      "  Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB)\n",
      "Requirement already satisfied: executing==1.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 44)) (1.2.0)\n",
      "Collecting fairscale==0.4.6\n",
      "  Downloading fairscale-0.4.6.tar.gz (248 kB)\n",
      "\u001b[K     |████████████████████████████████| 248 kB 72.7 MB/s \n",
      "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
      "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
      "  Installing backend dependencies ... \u001b[?25l\u001b[?25hdone\n",
      "    Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
      "Collecting fastjsonschema==2.16.3\n",
      "  Downloading fastjsonschema-2.16.3-py3-none-any.whl (23 kB)\n",
      "Collecting feedparser==6.0.8\n",
      "  Downloading feedparser-6.0.8-py3-none-any.whl (81 kB)\n",
      "\u001b[K     |████████████████████████████████| 81 kB 9.8 MB/s \n",
      "\u001b[?25hCollecting filelock==3.4.2\n",
      "  Downloading filelock-3.4.2-py3-none-any.whl (9.9 kB)\n",
      "Collecting flit_core==3.8.0\n",
      "  Downloading flit_core-3.8.0-py3-none-any.whl (62 kB)\n",
      "\u001b[K     |████████████████████████████████| 62 kB 1.3 MB/s \n",
      "\u001b[?25hCollecting fonttools==4.39.3\n",
      "  Downloading fonttools-4.39.3-py3-none-any.whl (1.0 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.0 MB 62.4 MB/s \n",
      "\u001b[?25hCollecting fqdn==1.5.1\n",
      "  Downloading fqdn-1.5.1-py3-none-any.whl (9.1 kB)\n",
      "Collecting frozenlist==1.3.0\n",
      "  Downloading frozenlist-1.3.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (156 kB)\n",
      "\u001b[K     |████████████████████████████████| 156 kB 71.7 MB/s \n",
      "\u001b[?25hCollecting fsspec==2022.1.0\n",
      "  Downloading fsspec-2022.1.0-py3-none-any.whl (133 kB)\n",
      "\u001b[K     |████████████████████████████████| 133 kB 83.6 MB/s \n",
      "\u001b[?25hCollecting future==0.18.2\n",
      "  Downloading future-0.18.2.tar.gz (829 kB)\n",
      "\u001b[K     |████████████████████████████████| 829 kB 69.2 MB/s \n",
      "\u001b[?25hCollecting gitdb==4.0.9\n",
      "  Downloading gitdb-4.0.9-py3-none-any.whl (63 kB)\n",
      "\u001b[K     |████████████████████████████████| 63 kB 1.8 MB/s \n",
      "\u001b[?25hCollecting GitPython==3.1.26\n",
      "  Downloading GitPython-3.1.26-py3-none-any.whl (180 kB)\n",
      "\u001b[K     |████████████████████████████████| 180 kB 71.4 MB/s \n",
      "\u001b[?25hCollecting google-api-core==2.5.0\n",
      "  Downloading google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)\n",
      "\u001b[K     |████████████████████████████████| 111 kB 79.2 MB/s \n",
      "\u001b[?25hCollecting google-auth==2.6.0\n",
      "  Downloading google_auth-2.6.0-py2.py3-none-any.whl (156 kB)\n",
      "\u001b[K     |████████████████████████████████| 156 kB 81.4 MB/s \n",
      "\u001b[?25hCollecting google-cloud-core==2.2.2\n",
      "  Downloading google_cloud_core-2.2.2-py2.py3-none-any.whl (29 kB)\n",
      "Collecting google-cloud-storage==1.44.0\n",
      "  Downloading google_cloud_storage-1.44.0-py2.py3-none-any.whl (106 kB)\n",
      "\u001b[K     |████████████████████████████████| 106 kB 71.3 MB/s \n",
      "\u001b[?25hCollecting google-crc32c==1.3.0\n",
      "  Downloading google_crc32c-1.3.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (36 kB)\n",
      "Collecting google-resumable-media==2.1.0\n",
      "  Downloading google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)\n",
      "\u001b[K     |████████████████████████████████| 75 kB 5.1 MB/s \n",
      "\u001b[?25hCollecting googleapis-common-protos==1.54.0\n",
      "  Downloading googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)\n",
      "\u001b[K     |████████████████████████████████| 207 kB 82.2 MB/s \n",
      "\u001b[?25hCollecting gpustat==1.1\n",
      "  Downloading gpustat-1.1.tar.gz (97 kB)\n",
      "\u001b[K     |████████████████████████████████| 97 kB 8.6 MB/s \n",
      "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
      "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
      "    Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
      "Collecting h5py==3.6.0\n",
      "  Downloading h5py-3.6.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)\n",
      "\u001b[K     |████████████████████████████████| 4.5 MB 58.7 MB/s \n",
      "\u001b[?25hCollecting higher==0.2.1\n",
      "  Downloading higher-0.2.1-py3-none-any.whl (27 kB)\n",
      "Collecting huggingface-hub==0.14.1\n",
      "  Downloading huggingface_hub-0.14.1-py3-none-any.whl (224 kB)\n",
      "\u001b[K     |████████████████████████████████| 224 kB 72.3 MB/s \n",
      "\u001b[?25hCollecting hydra-core==1.1.1\n",
      "  Downloading hydra_core-1.1.1-py3-none-any.whl (145 kB)\n",
      "\u001b[K     |████████████████████████████████| 145 kB 81.3 MB/s \n",
      "\u001b[?25hRequirement already satisfied: idna==3.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 69)) (3.4)\n",
      "Collecting importlib-metadata==6.3.0\n",
      "  Downloading importlib_metadata-6.3.0-py3-none-any.whl (22 kB)\n",
      "Collecting iniconfig==1.1.1\n",
      "  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)\n",
      "Collecting ipykernel==6.22.0\n",
      "  Downloading ipykernel-6.22.0-py3-none-any.whl (149 kB)\n",
      "\u001b[K     |████████████████████████████████| 149 kB 66.0 MB/s \n",
      "\u001b[?25hCollecting ipython==8.12.0\n",
      "  Downloading ipython-8.12.0-py3-none-any.whl (796 kB)\n",
      "\u001b[K     |████████████████████████████████| 796 kB 67.9 MB/s \n",
      "\u001b[?25hRequirement already satisfied: ipython-genutils==0.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 74)) (0.2.0)\n",
      "Collecting ipywidgets==8.0.6\n",
      "  Downloading ipywidgets-8.0.6-py3-none-any.whl (138 kB)\n",
      "\u001b[K     |████████████████████████████████| 138 kB 62.1 MB/s \n",
      "\u001b[?25hCollecting iso-639==0.4.5\n",
      "  Downloading iso-639-0.4.5.tar.gz (167 kB)\n",
      "\u001b[K     |████████████████████████████████| 167 kB 61.8 MB/s \n",
      "\u001b[?25hCollecting isoduration==20.11.0\n",
      "  Downloading isoduration-20.11.0-py3-none-any.whl (11 kB)\n",
      "Collecting jaraco.classes==3.2.1\n",
      "  Downloading jaraco.classes-3.2.1-py3-none-any.whl (5.6 kB)\n",
      "Collecting jaraco.collections==3.5.1\n",
      "  Downloading jaraco.collections-3.5.1-py3-none-any.whl (10 kB)\n",
      "Collecting jaraco.context==4.1.1\n",
      "  Downloading jaraco.context-4.1.1-py3-none-any.whl (4.4 kB)\n",
      "Collecting jaraco.functools==3.5.0\n",
      "  Downloading jaraco.functools-3.5.0-py3-none-any.whl (7.0 kB)\n",
      "Collecting jaraco.text==3.7.0\n",
      "  Downloading jaraco.text-3.7.0-py3-none-any.whl (8.6 kB)\n",
      "Requirement already satisfied: jedi==0.18.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 83)) (0.18.2)\n",
      "Requirement already satisfied: Jinja2==3.1.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 84)) (3.1.2)\n",
      "Collecting jmespath==0.10.0\n",
      "  Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)\n",
      "Collecting joblib==1.1.0\n",
      "  Downloading joblib-1.1.0-py2.py3-none-any.whl (306 kB)\n",
      "\u001b[K     |████████████████████████████████| 306 kB 61.1 MB/s \n",
      "\u001b[?25hCollecting jsonnet==0.18.0\n",
      "  Downloading jsonnet-0.18.0.tar.gz (592 kB)\n",
      "\u001b[K     |████████████████████████████████| 592 kB 76.8 MB/s \n",
      "\u001b[?25hCollecting jsonpointer==2.3\n",
      "  Downloading jsonpointer-2.3-py2.py3-none-any.whl (7.8 kB)\n",
      "Collecting jsonschema==4.17.3\n",
      "  Downloading jsonschema-4.17.3-py3-none-any.whl (90 kB)\n",
      "\u001b[K     |████████████████████████████████| 90 kB 10.7 MB/s \n",
      "\u001b[?25hCollecting jupyter==1.0.0\n",
      "  Using cached jupyter-1.0.0-py2.py3-none-any.whl (2.7 kB)\n",
      "Requirement already satisfied: jupyter-console==6.6.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 91)) (6.6.3)\n",
      "Collecting jupyter-events==0.6.3\n",
      "  Using cached jupyter_events-0.6.3-py3-none-any.whl (18 kB)\n",
      "Collecting jupyter_client==8.2.0\n",
      "  Downloading jupyter_client-8.2.0-py3-none-any.whl (103 kB)\n",
      "\u001b[K     |████████████████████████████████| 103 kB 76.8 MB/s \n",
      "\u001b[?25hCollecting jupyter_core==5.3.0\n",
      "  Downloading jupyter_core-5.3.0-py3-none-any.whl (93 kB)\n",
      "\u001b[K     |████████████████████████████████| 93 kB 1.8 MB/s \n",
      "\u001b[?25hCollecting jupyter_server==2.5.0\n",
      "  Downloading jupyter_server-2.5.0-py3-none-any.whl (366 kB)\n",
      "\u001b[K     |████████████████████████████████| 366 kB 78.0 MB/s \n",
      "\u001b[?25hRequirement already satisfied: jupyter_server_terminals==0.4.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 96)) (0.4.4)\n",
      "Requirement already satisfied: jupyterlab-pygments==0.2.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 97)) (0.2.2)\n",
      "Collecting jupyterlab-widgets==3.0.7\n",
      "  Downloading jupyterlab_widgets-3.0.7-py3-none-any.whl (198 kB)\n",
      "\u001b[K     |████████████████████████████████| 198 kB 82.2 MB/s \n",
      "\u001b[?25hCollecting kiwisolver==1.4.4\n",
      "  Downloading kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.6 MB 76.9 MB/s \n",
      "\u001b[?25hCollecting langcodes==3.3.0\n",
      "  Downloading langcodes-3.3.0-py3-none-any.whl (181 kB)\n",
      "\u001b[K     |████████████████████████████████| 181 kB 83.3 MB/s \n",
      "\u001b[?25hCollecting lmdb==1.3.0\n",
      "  Downloading lmdb-1.3.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (305 kB)\n",
      "\u001b[K     |████████████████████████████████| 305 kB 77.8 MB/s \n",
      "\u001b[?25hCollecting lxml==4.7.1\n",
      "  Downloading lxml-4.7.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.9 MB)\n",
      "\u001b[K     |████████████████████████████████| 6.9 MB 73.0 MB/s \n",
      "\u001b[?25hCollecting MarkupSafe==2.1.2\n",
      "  Downloading MarkupSafe-2.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)\n",
      "Collecting matplotlib==3.5.1\n",
      "  Downloading matplotlib-3.5.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (11.2 MB)\n",
      "\u001b[K     |████████████████████████████████| 11.2 MB 77.4 MB/s \n",
      "\u001b[?25hRequirement already satisfied: matplotlib-inline==0.1.6 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 105)) (0.1.6)\n",
      "Collecting mistune==2.0.5\n",
      "  Downloading mistune-2.0.5-py2.py3-none-any.whl (24 kB)\n",
      "Collecting more-itertools==8.12.0\n",
      "  Downloading more_itertools-8.12.0-py3-none-any.whl (54 kB)\n",
      "\u001b[K     |████████████████████████████████| 54 kB 3.4 MB/s \n",
      "\u001b[?25hCollecting multidict==6.0.2\n",
      "  Downloading multidict-6.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n",
      "\u001b[K     |████████████████████████████████| 114 kB 72.5 MB/s \n",
      "\u001b[?25hCollecting multiprocess==0.70.12.2\n",
      "  Downloading multiprocess-0.70.12.2-py39-none-any.whl (128 kB)\n",
      "\u001b[K     |████████████████████████████████| 128 kB 79.6 MB/s \n",
      "\u001b[?25hCollecting munch==2.5.0\n",
      "  Downloading munch-2.5.0-py2.py3-none-any.whl (10 kB)\n",
      "Collecting murmurhash==1.0.6\n",
      "  Downloading murmurhash-1.0.6-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (21 kB)\n",
      "Collecting mypy-extensions==0.4.3\n",
      "  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)\n",
      "Collecting nbclassic==0.5.5\n",
      "  Downloading nbclassic-0.5.5-py3-none-any.whl (10.0 MB)\n",
      "\u001b[K     |████████████████████████████████| 10.0 MB 77.8 MB/s \n",
      "\u001b[?25hCollecting nbclient==0.7.3\n",
      "  Downloading nbclient-0.7.3-py3-none-any.whl (72 kB)\n",
      "\u001b[K     |████████████████████████████████| 72 kB 1.0 MB/s \n",
      "\u001b[?25hCollecting nbconvert==7.3.1\n",
      "  Downloading nbconvert-7.3.1-py3-none-any.whl (284 kB)\n",
      "\u001b[K     |████████████████████████████████| 284 kB 78.5 MB/s \n",
      "\u001b[?25hCollecting nbformat==5.8.0\n",
      "  Downloading nbformat-5.8.0-py3-none-any.whl (77 kB)\n",
      "\u001b[K     |████████████████████████████████| 77 kB 6.2 MB/s \n",
      "\u001b[?25hRequirement already satisfied: nest-asyncio==1.5.6 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 117)) (1.5.6)\n",
      "Collecting nltk==3.6.5\n",
      "  Downloading nltk-3.6.5-py3-none-any.whl (1.5 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.5 MB 74.3 MB/s \n",
      "\u001b[?25hCollecting notebook==6.5.4\n",
      "  Using cached notebook-6.5.4-py3-none-any.whl (529 kB)\n",
      "Collecting notebook_shim==0.2.2\n",
      "  Downloading notebook_shim-0.2.2-py3-none-any.whl (13 kB)\n",
      "Collecting numpy==1.22.1\n",
      "  Downloading numpy-1.22.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)\n",
      "\u001b[K     |████████████████████████████████| 16.8 MB 72.3 MB/s \n",
      "\u001b[?25hCollecting nvidia-ml-py==11.525.112\n",
      "  Downloading nvidia_ml_py-11.525.112-py3-none-any.whl (35 kB)\n",
      "Collecting omegaconf==2.1.1\n",
      "  Downloading omegaconf-2.1.1-py3-none-any.whl (74 kB)\n",
      "\u001b[K     |████████████████████████████████| 74 kB 3.7 MB/s \n",
      "\u001b[?25hRequirement already satisfied: packaging==23.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 124)) (23.1)\n",
      "Collecting pandas==1.4.0\n",
      "  Downloading pandas-1.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)\n",
      "\u001b[K     |████████████████████████████████| 11.7 MB 65.2 MB/s \n",
      "\u001b[?25hRequirement already satisfied: pandocfilters==1.5.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 126)) (1.5.0)\n",
      "Requirement already satisfied: parso==0.8.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 127)) (0.8.3)\n",
      "Collecting pathspec==0.9.0\n",
      "  Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)\n",
      "Collecting pathtools==0.1.2\n",
      "  Downloading pathtools-0.1.2.tar.gz (11 kB)\n",
      "Collecting pathy==0.6.1\n",
      "  Downloading pathy-0.6.1-py3-none-any.whl (42 kB)\n",
      "\u001b[K     |████████████████████████████████| 42 kB 1.5 MB/s \n",
      "\u001b[?25hCollecting patternfork-nosql==3.6\n",
      "  Downloading patternfork_nosql-3.6.tar.gz (22.3 MB)\n",
      "\u001b[K     |████████████████████████████████| 22.3 MB 1.4 MB/s \n",
      "\u001b[?25hCollecting pdfminer.six==20211012\n",
      "  Downloading pdfminer.six-20211012-py3-none-any.whl (5.6 MB)\n",
      "\u001b[K     |████████████████████████████████| 5.6 MB 50.8 MB/s \n",
      "\u001b[?25hRequirement already satisfied: pexpect==4.8.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 133)) (4.8.0)\n",
      "Requirement already satisfied: pickleshare==0.7.5 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 134)) (0.7.5)\n",
      "Collecting Pillow==9.0.0\n",
      "  Downloading Pillow-9.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)\n",
      "\u001b[K     |████████████████████████████████| 4.3 MB 76.2 MB/s \n",
      "\u001b[?25hCollecting platformdirs==3.2.0\n",
      "  Downloading platformdirs-3.2.0-py3-none-any.whl (14 kB)\n",
      "Collecting pluggy==1.0.0\n",
      "  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)\n",
      "Collecting portend==3.1.0\n",
      "  Downloading portend-3.1.0-py3-none-any.whl (5.3 kB)\n",
      "Collecting preshed==3.0.6\n",
      "  Downloading preshed-3.0.6-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (128 kB)\n",
      "\u001b[K     |████████████████████████████████| 128 kB 78.4 MB/s \n",
      "\u001b[?25hCollecting prometheus-client==0.16.0\n",
      "  Downloading prometheus_client-0.16.0-py3-none-any.whl (122 kB)\n",
      "\u001b[K     |████████████████████████████████| 122 kB 82.0 MB/s \n",
      "\u001b[?25hCollecting promise==2.3\n",
      "  Downloading promise-2.3.tar.gz (19 kB)\n",
      "Collecting prompt-toolkit==3.0.38\n",
      "  Downloading prompt_toolkit-3.0.38-py3-none-any.whl (385 kB)\n",
      "\u001b[K     |████████████████████████████████| 385 kB 74.2 MB/s \n",
      "\u001b[?25hCollecting protobuf==3.19.4\n",
      "  Downloading protobuf-3.19.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.1 MB 74.3 MB/s \n",
      "\u001b[?25hCollecting psutil==5.9.4\n",
      "  Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)\n",
      "\u001b[K     |████████████████████████████████| 280 kB 77.7 MB/s \n",
      "\u001b[?25hRequirement already satisfied: ptyprocess==0.7.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 145)) (0.7.0)\n",
      "Requirement already satisfied: pure-eval==0.2.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 146)) (0.2.2)\n",
      "Collecting py==1.11.0\n",
      "  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)\n",
      "\u001b[K     |████████████████████████████████| 98 kB 10.1 MB/s \n",
      "\u001b[?25hCollecting pyarrow==6.0.1\n",
      "  Downloading pyarrow-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25.6 MB)\n",
      "\u001b[K     |████████████████████████████████| 25.6 MB 103.2 MB/s \n",
      "\u001b[?25hCollecting pyasn1==0.4.8\n",
      "  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)\n",
      "\u001b[K     |████████████████████████████████| 77 kB 6.2 MB/s \n",
      "\u001b[?25hCollecting pyasn1-modules==0.2.8\n",
      "  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)\n",
      "\u001b[K     |████████████████████████████████| 155 kB 80.4 MB/s \n",
      "\u001b[?25hRequirement already satisfied: pycparser==2.21 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 151)) (2.21)\n",
      "Collecting pydantic==1.8.2\n",
      "  Downloading pydantic-1.8.2-cp39-cp39-manylinux2014_x86_64.whl (11.3 MB)\n",
      "\u001b[K     |████████████████████████████████| 11.3 MB 62.8 MB/s \n",
      "\u001b[?25hCollecting Pygments==2.15.0\n",
      "  Downloading Pygments-2.15.0-py3-none-any.whl (1.1 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.1 MB 65.3 MB/s \n",
      "\u001b[?25hCollecting pyparsing==3.0.7\n",
      "  Downloading pyparsing-3.0.7-py3-none-any.whl (98 kB)\n",
      "\u001b[K     |████████████████████████████████| 98 kB 9.5 MB/s \n",
      "\u001b[?25hCollecting pyrsistent==0.19.3\n",
      "  Downloading pyrsistent-0.19.3-py3-none-any.whl (57 kB)\n",
      "\u001b[K     |████████████████████████████████| 57 kB 6.3 MB/s \n",
      "\u001b[?25hCollecting pytest==6.2.5\n",
      "  Downloading pytest-6.2.5-py3-none-any.whl (280 kB)\n",
      "\u001b[K     |████████████████████████████████| 280 kB 73.1 MB/s \n",
      "\u001b[?25hRequirement already satisfied: python-dateutil==2.8.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 157)) (2.8.2)\n",
      "Collecting python-docx==0.8.11\n",
      "  Downloading python-docx-0.8.11.tar.gz (5.6 MB)\n",
      "\u001b[K     |████████████████████████████████| 5.6 MB 63.3 MB/s \n",
      "\u001b[?25hCollecting python-dotenv==0.19.2\n",
      "  Downloading python_dotenv-0.19.2-py2.py3-none-any.whl (17 kB)\n",
      "Requirement already satisfied: python-json-logger==2.0.7 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 160)) (2.0.7)\n",
      "Collecting pytz==2021.3\n",
      "  Downloading pytz-2021.3-py2.py3-none-any.whl (503 kB)\n",
      "\u001b[K     |████████████████████████████████| 503 kB 70.9 MB/s \n",
      "\u001b[?25hRequirement already satisfied: PyYAML==6.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 162)) (6.0)\n",
      "Collecting pyzmq==25.0.2\n",
      "  Downloading pyzmq-25.0.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.1 MB 64.3 MB/s \n",
      "\u001b[?25hCollecting qtconsole==5.4.2\n",
      "  Downloading qtconsole-5.4.2-py3-none-any.whl (121 kB)\n",
      "\u001b[K     |████████████████████████████████| 121 kB 67.5 MB/s \n",
      "\u001b[?25hRequirement already satisfied: QtPy==2.3.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 165)) (2.3.1)\n",
      "Collecting regex==2022.1.18\n",
      "  Downloading regex-2022.1.18-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (763 kB)\n",
      "\u001b[K     |████████████████████████████████| 763 kB 58.0 MB/s \n",
      "\u001b[?25hCollecting requests==2.30.0\n",
      "  Downloading requests-2.30.0-py3-none-any.whl (62 kB)\n",
      "\u001b[K     |████████████████████████████████| 62 kB 1.0 MB/s \n",
      "\u001b[?25hRequirement already satisfied: rfc3339-validator==0.1.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 168)) (0.1.4)\n",
      "Requirement already satisfied: rfc3986-validator==0.1.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 169)) (0.1.1)\n",
      "Collecting rich==12.6.0\n",
      "  Downloading rich-12.6.0-py3-none-any.whl (237 kB)\n",
      "\u001b[K     |████████████████████████████████| 237 kB 60.2 MB/s \n",
      "\u001b[?25hCollecting rsa==4.8\n",
      "  Downloading rsa-4.8-py3-none-any.whl (39 kB)\n",
      "Collecting ruamel.yaml.clib==0.2.7\n",
      "  Downloading ruamel.yaml.clib-0.2.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (519 kB)\n",
      "\u001b[K     |████████████████████████████████| 519 kB 81.6 MB/s \n",
      "\u001b[?25hCollecting s3transfer==0.5.1\n",
      "  Downloading s3transfer-0.5.1-py3-none-any.whl (79 kB)\n",
      "\u001b[K     |████████████████████████████████| 79 kB 8.9 MB/s \n",
      "\u001b[?25hCollecting sacremoses==0.0.47\n",
      "  Downloading sacremoses-0.0.47-py2.py3-none-any.whl (895 kB)\n",
      "\u001b[K     |████████████████████████████████| 895 kB 62.2 MB/s \n",
      "\u001b[?25hCollecting safetensors==0.3.1\n",
      "  Downloading safetensors-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.3 MB 50.4 MB/s \n",
      "\u001b[?25hCollecting scikit-learn==1.0.2\n",
      "  Downloading scikit_learn-1.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.4 MB)\n",
      "\u001b[K     |████████████████████████████████| 26.4 MB 1.3 MB/s \n",
      "\u001b[?25hCollecting scipy==1.7.3\n",
      "  Downloading scipy-1.7.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.8 MB)\n",
      "\u001b[K     |████████████████████████████████| 39.8 MB 1.6 MB/s \n",
      "\u001b[?25hCollecting Send2Trash==1.8.0\n",
      "  Downloading Send2Trash-1.8.0-py3-none-any.whl (18 kB)\n",
      "Collecting sentence-transformers==2.2.2\n",
      "  Downloading sentence-transformers-2.2.2.tar.gz (85 kB)\n",
      "\u001b[K     |████████████████████████████████| 85 kB 6.0 MB/s \n",
      "\u001b[?25hCollecting sentencepiece==0.1.96\n",
      "  Downloading sentencepiece-0.1.96-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.2 MB 59.7 MB/s \n",
      "\u001b[?25hCollecting sentry-sdk==1.5.4\n",
      "  Downloading sentry_sdk-1.5.4-py2.py3-none-any.whl (143 kB)\n",
      "\u001b[K     |████████████████████████████████| 143 kB 81.8 MB/s \n",
      "\u001b[?25hCollecting sgmllib3k==1.0.0\n",
      "  Downloading sgmllib3k-1.0.0.tar.gz (5.8 kB)\n",
      "Collecting shortuuid==1.0.8\n",
      "  Downloading shortuuid-1.0.8-py3-none-any.whl (9.5 kB)\n",
      "Requirement already satisfied: six==1.16.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 184)) (1.16.0)\n",
      "Collecting sklearn==0.0\n",
      "  Downloading sklearn-0.0.tar.gz (1.1 kB)\n",
      "Collecting smart-open==5.2.1\n",
      "  Downloading smart_open-5.2.1-py3-none-any.whl (58 kB)\n",
      "\u001b[K     |████████████████████████████████| 58 kB 6.7 MB/s \n",
      "\u001b[?25hCollecting smmap==5.0.0\n",
      "  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)\n",
      "Requirement already satisfied: sniffio==1.3.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 188)) (1.3.0)\n",
      "Collecting soupsieve==2.4\n",
      "  Downloading soupsieve-2.4-py3-none-any.whl (37 kB)\n",
      "Collecting spacy==3.2.1\n",
      "  Downloading spacy-3.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.1 MB)\n",
      "\u001b[K     |████████████████████████████████| 6.1 MB 72.6 MB/s \n",
      "\u001b[?25hCollecting spacy-legacy==3.0.8\n",
      "  Downloading spacy_legacy-3.0.8-py2.py3-none-any.whl (14 kB)\n",
      "Collecting spacy-loggers==1.0.1\n",
      "  Downloading spacy_loggers-1.0.1-py3-none-any.whl (7.0 kB)\n",
      "Collecting srsly==2.4.2\n",
      "  Downloading srsly-2.4.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (452 kB)\n",
      "\u001b[K     |████████████████████████████████| 452 kB 79.8 MB/s \n",
      "\u001b[?25hRequirement already satisfied: stack-data==0.6.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 194)) (0.6.2)\n",
      "Collecting tempora==5.0.1\n",
      "  Downloading tempora-5.0.1-py3-none-any.whl (15 kB)\n",
      "Collecting tensorboardX==2.4.1\n",
      "  Downloading tensorboardX-2.4.1-py2.py3-none-any.whl (124 kB)\n",
      "\u001b[K     |████████████████████████████████| 124 kB 73.9 MB/s \n",
      "\u001b[?25hCollecting termcolor==1.1.0\n",
      "  Downloading termcolor-1.1.0.tar.gz (3.9 kB)\n",
      "Requirement already satisfied: terminado==0.17.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 198)) (0.17.1)\n",
      "Collecting testpath==0.5.0\n",
      "  Downloading testpath-0.5.0-py3-none-any.whl (84 kB)\n",
      "\u001b[K     |████████████████████████████████| 84 kB 4.2 MB/s \n",
      "\u001b[?25hCollecting thinc==8.0.13\n",
      "  Downloading thinc-8.0.13-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (635 kB)\n",
      "\u001b[K     |████████████████████████████████| 635 kB 75.6 MB/s \n",
      "\u001b[?25hCollecting threadpoolctl==3.1.0\n",
      "  Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)\n",
      "Requirement already satisfied: tinycss2==1.2.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 202)) (1.2.1)\n",
      "Collecting tokenizers==0.12.1\n",
      "  Downloading tokenizers-0.12.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB)\n",
      "\u001b[K     |████████████████████████████████| 6.6 MB 75.9 MB/s \n",
      "\u001b[?25hCollecting toml==0.10.2\n",
      "  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\n",
      "Collecting tomli==2.0.0\n",
      "  Downloading tomli-2.0.0-py3-none-any.whl (12 kB)\n",
      "Collecting torch==1.10.2\n",
      "  Downloading torch-1.10.2-cp39-cp39-manylinux1_x86_64.whl (881.9 MB)\n",
      "\u001b[K     |████████████████████████████████| 881.9 MB 14 kB/s \n",
      "\u001b[?25hCollecting torchvision==0.11.3\n",
      "  Downloading torchvision-0.11.3-cp39-cp39-manylinux1_x86_64.whl (23.2 MB)\n",
      "\u001b[K     |████████████████████████████████| 23.2 MB 1.1 MB/s \n",
      "\u001b[?25hCollecting tornado==6.2\n",
      "  Downloading tornado-6.2-cp37-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (423 kB)\n",
      "\u001b[K     |████████████████████████████████| 423 kB 82.2 MB/s \n",
      "\u001b[?25hCollecting tqdm==4.62.3\n",
      "  Downloading tqdm-4.62.3-py2.py3-none-any.whl (76 kB)\n",
      "\u001b[K     |████████████████████████████████| 76 kB 5.8 MB/s \n",
      "\u001b[?25hRequirement already satisfied: traitlets==5.9.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 210)) (5.9.0)\n",
      "Collecting transformers==4.30.1\n",
      "  Downloading transformers-4.30.1-py3-none-any.whl (7.2 MB)\n",
      "\u001b[K     |████████████████████████████████| 7.2 MB 72.7 MB/s \n",
      "\u001b[?25hCollecting typer==0.4.2\n",
      "  Downloading typer-0.4.2-py3-none-any.whl (27 kB)\n",
      "Collecting typing_extensions==4.4.0\n",
      "  Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)\n",
      "Collecting uri-template==1.2.0\n",
      "  Downloading uri_template-1.2.0-py3-none-any.whl (10 kB)\n",
      "Collecting urllib3==1.26.8\n",
      "  Downloading urllib3-1.26.8-py2.py3-none-any.whl (138 kB)\n",
      "\u001b[K     |████████████████████████████████| 138 kB 75.1 MB/s \n",
      "\u001b[?25hCollecting wandb==0.12.10\n",
      "  Downloading wandb-0.12.10-py2.py3-none-any.whl (1.7 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.7 MB 75.9 MB/s \n",
      "\u001b[?25hCollecting wasabi==0.9.0\n",
      "  Downloading wasabi-0.9.0-py3-none-any.whl (25 kB)\n",
      "Requirement already satisfied: wcwidth==0.2.6 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 218)) (0.2.6)\n",
      "Collecting webcolors==1.13\n",
      "  Downloading webcolors-1.13-py3-none-any.whl (14 kB)\n",
      "Requirement already satisfied: webencodings==0.5.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 220)) (0.5.1)\n",
      "Collecting websocket-client==1.5.1\n",
      "  Downloading websocket_client-1.5.1-py3-none-any.whl (55 kB)\n",
      "\u001b[K     |████████████████████████████████| 55 kB 4.6 MB/s \n",
      "\u001b[?25hCollecting widgetsnbextension==4.0.7\n",
      "  Downloading widgetsnbextension-4.0.7-py3-none-any.whl (2.1 MB)\n",
      "\u001b[K     |████████████████████████████████| 2.1 MB 72.4 MB/s \n",
      "\u001b[?25hCollecting xxhash==2.0.2\n",
      "  Downloading xxhash-2.0.2-cp39-cp39-manylinux2010_x86_64.whl (243 kB)\n",
      "\u001b[K     |████████████████████████████████| 243 kB 81.4 MB/s \n",
      "\u001b[?25hCollecting yarl==1.7.2\n",
      "  Downloading yarl-1.7.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (304 kB)\n",
      "\u001b[K     |████████████████████████████████| 304 kB 77.9 MB/s \n",
      "\u001b[?25hCollecting yaspin==2.1.0\n",
      "  Downloading yaspin-2.1.0-py3-none-any.whl (18 kB)\n",
      "Collecting zc.lockfile==2.0\n",
      "  Downloading zc.lockfile-2.0-py2.py3-none-any.whl (9.7 kB)\n",
      "Collecting zipp==3.15.0\n",
      "  Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB)\n",
      "Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from spacy==3.2.1->-r requirements.txt (line 190)) (45.2.0)\n",
      "Building wheels for collected packages: antlr4-python3-runtime, checklist, fairscale, future, gpustat, iso-639, jsonnet, pathtools, patternfork-nosql, promise, python-docx, sentence-transformers, sgmllib3k, sklearn, termcolor\n",
      "  Building wheel for antlr4-python3-runtime (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.8-py3-none-any.whl size=141230 sha256=51727e1658f33c7f7bf71f4a3d7f637a385ee552e9340e8460b15e7fb34116e8\n",
      "  Stored in directory: /root/.cache/pip/wheels/42/3c/ae/14db087e6018de74810afe32eb6ac890ef9c68ba19b00db97a\n",
      "  Building wheel for checklist (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for checklist: filename=checklist-0.0.11-py3-none-any.whl size=12165625 sha256=862b330f4abcaa93f6f416bb0291d04209e1d14ffbed2798a92a0af4c8dc34ec\n",
      "  Stored in directory: /root/.cache/pip/wheels/70/48/f4/983e6cfe66fed9979451c546178d6d6f92ba1d6dc5e6add3e4\n",
      "  Building wheel for fairscale (PEP 517) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for fairscale: filename=fairscale-0.4.6-py3-none-any.whl size=307224 sha256=a2a290a235f667a968d205ce6a3eff1db052093743350366e7456bdc47e0ffb1\n",
      "  Stored in directory: /root/.cache/pip/wheels/59/b9/f8/08050f4b5734886ee64f25d1725e9c1e0a2bd19482b0385f25\n",
      "  Building wheel for future (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=e27d4687bc0e4fb3beae944df2cd221e45be361161f0734fd9e617e781a9c319\n",
      "  Stored in directory: /root/.cache/pip/wheels/2f/a0/d3/4030d9f80e6b3be787f19fc911b8e7aa462986a40ab1e4bb94\n",
      "  Building wheel for gpustat (PEP 517) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for gpustat: filename=gpustat-1.1-py3-none-any.whl size=26280 sha256=65601050a2d5fc3be8a04818903807c823a0cfca86fafb650212b4406834f70f\n",
      "  Stored in directory: /root/.cache/pip/wheels/91/f0/b3/8566d6821307110981a5db015cbf8fd88697446f81e5f40a27\n",
      "  Building wheel for iso-639 (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for iso-639: filename=iso_639-0.4.5-py3-none-any.whl size=169061 sha256=07fbd0091f68f410f67b930d3459fa2d13a6aac32aecc050d6846dfca8d42231\n",
      "  Stored in directory: /root/.cache/pip/wheels/43/3f/de/07f35ac2a2cd11ff30224e3fc6fbf458d7fc95effb1f673431\n",
      "  Building wheel for jsonnet (setup.py) ... \u001b[?25lerror\n",
      "\u001b[31m  ERROR: Failed building wheel for jsonnet\u001b[0m\n",
      "\u001b[?25h  Running setup.py clean for jsonnet\n",
      "  Building wheel for pathtools (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8784 sha256=fdf06c3d97400dfa4b4b3b78c80dfb17e03629718afff13df1112f89641e0902\n",
      "  Stored in directory: /root/.cache/pip/wheels/b7/0a/67/ada2a22079218c75a88361c0782855cc72aebc4d18d0289d05\n",
      "  Building wheel for patternfork-nosql (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for patternfork-nosql: filename=patternfork_nosql-3.6-py3-none-any.whl size=22332805 sha256=4f2b198153395c18d8a793ab11de87b3fe3fe07a7c8bba158efaa3ecd798524c\n",
      "  Stored in directory: /root/.cache/pip/wheels/b4/f8/c1/165dbb1ce03c55f2a5c7e075ef9c1c6ff71b3620fb4dbcda38\n",
      "  Building wheel for promise (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for promise: filename=promise-2.3-py3-none-any.whl size=21493 sha256=9fd09f3618907b15194115caa26363ca24ba2273d2ed218020682f7c711e6bc0\n",
      "  Stored in directory: /root/.cache/pip/wheels/e1/e8/83/ddea66100678d139b14bc87692ece57c6a2a937956d2532608\n",
      "  Building wheel for python-docx (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for python-docx: filename=python_docx-0.8.11-py3-none-any.whl size=184600 sha256=bd17dfd62990c036681395cc726fd0c8d0f05943475f03c0fe310d65893fb471\n",
      "  Stored in directory: /root/.cache/pip/wheels/83/8b/7c/09ae60c42c7ba4ed2dddaf2b8b9186cb105255856d6ed3dba5\n",
      "  Building wheel for sentence-transformers (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125922 sha256=5e54e526a6bb0103009c7f2085caeac205eab786d774e9e54aae30f98f84c755\n",
      "  Stored in directory: /root/.cache/pip/wheels/71/67/06/162a3760c40d74dd40bc855d527008d26341c2b0ecf3e8e11f\n",
      "  Building wheel for sgmllib3k (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for sgmllib3k: filename=sgmllib3k-1.0.0-py3-none-any.whl size=6064 sha256=9bda2f8fd5c067d508e01819257f299961d97ec9c23f4d3e4d0627867fb4d48a\n",
      "  Stored in directory: /root/.cache/pip/wheels/65/7a/a7/78c287f64e401255dff4c13fdbc672fed5efbfd21c530114e1\n",
      "  Building wheel for sklearn (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for sklearn: filename=sklearn-0.0-py2.py3-none-any.whl size=1315 sha256=700aa277398852d228ba14a127e90d912b3a31b0357613708108ab68533c075c\n",
      "  Stored in directory: /root/.cache/pip/wheels/e4/7b/98/b6466d71b8d738a0c547008b9eb39bf8676d1ff6ca4b22af1c\n",
      "  Building wheel for termcolor (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4830 sha256=bc752f4a62b57c8c04180ea7dfe8989cf41933ac13bcbe7cad0916b0d4d3573d\n",
      "  Stored in directory: /root/.cache/pip/wheels/b6/0d/90/0d1bbd99855f99cb2f6c2e5ff96f8023fad8ec367695f7d72d\n",
      "Successfully built antlr4-python3-runtime checklist fairscale future gpustat iso-639 pathtools patternfork-nosql promise python-docx sentence-transformers sgmllib3k sklearn termcolor\n",
      "Failed to build jsonnet\n",
      "Installing collected packages: frozenlist, aiosignal, multidict, async-timeout, attrs, yarl, charset-normalizer, aiohttp, antlr4-python3-runtime, anyio, cffi, argon2-cffi-bindings, argon2-cffi, arrow, asttokens, backcall, backports.csv, base58, soupsieve, beautifulsoup4, mypy-extensions, platformdirs, typing-extensions, click, pathspec, tomli, black, bleach, blessed, numpy, blis, jmespath, urllib3, botocore, s3transfer, boto3, cachetools, catalogue, certifi, chardet, dill, widgetsnbextension, psutil, pyzmq, prompt-toolkit, Pygments, ipython, zipp, importlib-metadata, tornado, jupyter-core, jupyter-client, ipykernel, jupyterlab-widgets, ipywidgets, iso-639, fastjsonschema, pyrsistent, jsonschema, nbformat, nbclient, MarkupSafe, mistune, nbconvert, jupyter-events, websocket-client, Send2Trash, prometheus-client, jupyter-server, notebook-shim, nbclassic, notebook, qtconsole, jupyter, munch, pytz, more-itertools, jaraco.functools, tempora, portend, cheroot, zc.lockfile, jaraco.classes, jaraco.context, jaraco.text, jaraco.collections, CherryPy, sgmllib3k, feedparser, future, lxml, regex, joblib, tqdm, nltk, cryptography, pdfminer.six, python-docx, requests, scipy, patternfork-nosql, wasabi, spacy-loggers, cymem, murmurhash, preshed, langcodes, srsly, pydantic, thinc, spacy-legacy, smart-open, typer, pathy, spacy, filelock, fsspec, huggingface-hub, safetensors, tokenizers, transformers, checklist, commonmark, cycler, xxhash, pyarrow, multiprocess, pandas, datasets, docker-pycreds, einops, entrypoints, torch, fairscale, flit-core, fonttools, fqdn, smmap, gitdb, GitPython, pyasn1, pyasn1-modules, rsa, google-auth, protobuf, googleapis-common-protos, google-api-core, google-cloud-core, google-crc32c, google-resumable-media, google-cloud-storage, nvidia-ml-py, gpustat, h5py, higher, omegaconf, hydra-core, iniconfig, isoduration, jsonnet, jsonpointer, kiwisolver, lmdb, pyparsing, Pillow, matplotlib, pathtools, pluggy, promise, py, toml, pytest, python-dotenv, rich, ruamel.yaml.clib, sacremoses, threadpoolctl, scikit-learn, sentencepiece, torchvision, sentence-transformers, sentry-sdk, shortuuid, sklearn, tensorboardX, termcolor, testpath, uri-template, yaspin, wandb, webcolors\n",
      "  Attempting uninstall: platformdirs\n",
      "    Found existing installation: platformdirs 3.9.1\n",
      "    Uninstalling platformdirs-3.9.1:\n",
      "      Successfully uninstalled platformdirs-3.9.1\n",
      "  Attempting uninstall: typing-extensions\n",
      "    Found existing installation: typing-extensions 4.7.1\n",
      "    Uninstalling typing-extensions-4.7.1:\n",
      "      Successfully uninstalled typing-extensions-4.7.1\n",
      "  Attempting uninstall: psutil\n",
      "    Found existing installation: psutil 5.9.5\n",
      "    Uninstalling psutil-5.9.5:\n",
      "      Successfully uninstalled psutil-5.9.5\n",
      "  Attempting uninstall: pyzmq\n",
      "    Found existing installation: pyzmq 25.1.0\n",
      "    Uninstalling pyzmq-25.1.0:\n",
      "      Successfully uninstalled pyzmq-25.1.0\n",
      "  Attempting uninstall: prompt-toolkit\n",
      "    Found existing installation: prompt-toolkit 3.0.39\n",
      "    Uninstalling prompt-toolkit-3.0.39:\n",
      "      Successfully uninstalled prompt-toolkit-3.0.39\n",
      "  Attempting uninstall: Pygments\n",
      "    Found existing installation: Pygments 2.15.1\n",
      "    Uninstalling Pygments-2.15.1:\n",
      "      Successfully uninstalled Pygments-2.15.1\n",
      "  Attempting uninstall: zipp\n",
      "    Found existing installation: zipp 3.16.2\n",
      "    Uninstalling zipp-3.16.2:\n",
      "      Successfully uninstalled zipp-3.16.2\n",
      "  Attempting uninstall: tornado\n",
      "    Found existing installation: tornado 6.3.2\n",
      "    Uninstalling tornado-6.3.2:\n",
      "      Successfully uninstalled tornado-6.3.2\n",
      "  Attempting uninstall: jupyter-core\n",
      "    Found existing installation: jupyter-core 5.3.1\n",
      "    Uninstalling jupyter-core-5.3.1:\n",
      "      Successfully uninstalled jupyter-core-5.3.1\n",
      "  Attempting uninstall: jupyter-client\n",
      "    Found existing installation: jupyter-client 8.3.0\n",
      "    Uninstalling jupyter-client-8.3.0:\n",
      "      Successfully uninstalled jupyter-client-8.3.0\n",
      "  Attempting uninstall: MarkupSafe\n",
      "    Found existing installation: MarkupSafe 2.1.3\n",
      "    Uninstalling MarkupSafe-2.1.3:\n",
      "      Successfully uninstalled MarkupSafe-2.1.3\n",
      "    Running setup.py install for jsonnet ... \u001b[?25l\u001b[?25herror\n",
      "\u001b[31mERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '\"'\"'/tmp/pip-install-kpd3f5zf/jsonnet/setup.py'\"'\"'; __file__='\"'\"'/tmp/pip-install-kpd3f5zf/jsonnet/setup.py'\"'\"';f=getattr(tokenize, '\"'\"'open'\"'\"', open)(__file__);code=f.read().replace('\"'\"'\\r\\n'\"'\"', '\"'\"'\\n'\"'\"');f.close();exec(compile(code, __file__, '\"'\"'exec'\"'\"'))' install --record /tmp/pip-record-lwkopa6a/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/jsonnet Check the logs for full command output.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!apt-get install python3-pip\n",
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "A0eGk7gM_wg4",
    "outputId": "0f3fc28c-641b-49b9-8983-b8269fca2443"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Package                  Version\n",
      "------------------------ ----------\n",
      "accelerate               1.0.1\n",
      "aiohappyeyeballs         2.4.3\n",
      "aiohttp                  3.10.10\n",
      "aiosignal                1.3.1\n",
      "antlr4-python3-runtime   4.8\n",
      "asttokens                2.4.1\n",
      "async-timeout            4.0.3\n",
      "attrs                    24.2.0\n",
      "blessed                  1.20.0\n",
      "certifi                  2024.8.30\n",
      "charset-normalizer       3.4.0\n",
      "click                    8.1.7\n",
      "cmake                    3.30.5\n",
      "comm                     0.2.2\n",
      "cycler                   0.12.1\n",
      "datasets                 1.18.3\n",
      "debugpy                  1.8.7\n",
      "decorator                5.1.1\n",
      "dill                     0.3.9\n",
      "einops                   0.4.0\n",
      "exceptiongroup           1.2.2\n",
      "executing                2.1.0\n",
      "fairscale                0.4.13\n",
      "filelock                 3.16.1\n",
      "fonttools                4.54.1\n",
      "frozenlist               1.5.0\n",
      "fsspec                   2024.10.0\n",
      "gpustat                  1.1\n",
      "higher                   0.2.1\n",
      "huggingface-hub          0.25.1\n",
      "hydra-core               1.1.1\n",
      "idna                     3.10\n",
      "importlib-metadata       6.3.0\n",
      "iopath                   0.1.10\n",
      "ipykernel                6.29.5\n",
      "ipython                  8.18.1\n",
      "ipywidgets               8.1.5\n",
      "jedi                     0.19.1\n",
      "Jinja2                   3.1.4\n",
      "joblib                   1.4.2\n",
      "jupyter_client           8.6.3\n",
      "jupyter_core             5.7.2\n",
      "jupyterlab_widgets       3.0.13\n",
      "kiwisolver               1.4.7\n",
      "lit                      18.1.8\n",
      "MarkupSafe               3.0.2\n",
      "matplotlib               3.5.1\n",
      "matplotlib-inline        0.1.7\n",
      "mpmath                   1.3.0\n",
      "multidict                6.1.0\n",
      "multiprocess             0.70.17\n",
      "nest_asyncio             1.6.0\n",
      "networkx                 3.2.1\n",
      "nltk                     3.6.5\n",
      "numpy                    1.22.1\n",
      "nvidia-cublas-cu11       11.10.3.66\n",
      "nvidia-cuda-cupti-cu11   11.7.101\n",
      "nvidia-cuda-nvrtc-cu11   11.7.99\n",
      "nvidia-cuda-runtime-cu11 11.7.99\n",
      "nvidia-cudnn-cu11        8.5.0.96\n",
      "nvidia-cufft-cu11        10.9.0.58\n",
      "nvidia-curand-cu11       10.2.10.91\n",
      "nvidia-cusolver-cu11     11.4.0.1\n",
      "nvidia-cusparse-cu11     11.7.4.91\n",
      "nvidia-ml-py             12.560.30\n",
      "nvidia-nccl-cu11         2.14.3\n",
      "nvidia-nvtx-cu11         11.7.91\n",
      "omegaconf                2.1.1\n",
      "openai                   0.27.9\n",
      "opencv-python            4.8.0.76\n",
      "packaging                24.1\n",
      "pandas                   1.4.0\n",
      "parso                    0.8.4\n",
      "peft                     0.7.1\n",
      "pexpect                  4.9.0\n",
      "pickleshare              0.7.5\n",
      "pillow                   11.0.0\n",
      "pip                      24.2\n",
      "platformdirs             4.3.6\n",
      "portalocker              2.10.1\n",
      "progressbar2             4.5.0\n",
      "prompt_toolkit           3.0.48\n",
      "propcache                0.2.0\n",
      "psutil                   6.0.0\n",
      "ptyprocess               0.7.0\n",
      "pure_eval                0.2.3\n",
      "pyarrow                  17.0.0\n",
      "Pygments                 2.18.0\n",
      "pyparsing                3.2.0\n",
      "python-dateutil          2.9.0\n",
      "python-utils             3.9.0\n",
      "pytz                     2024.2\n",
      "PyYAML                   6.0\n",
      "pyzmq                    26.2.0\n",
      "regex                    2024.9.11\n",
      "requests                 2.32.3\n",
      "safetensors              0.4.5\n",
      "scikit-learn             1.0.2\n",
      "scipy                    1.7.3\n",
      "sentence-transformers    2.2.2\n",
      "sentencepiece            0.2.0\n",
      "setuptools               75.1.0\n",
      "six                      1.16.0\n",
      "stack-data               0.6.2\n",
      "sympy                    1.13.3\n",
      "threadpoolctl            3.5.0\n",
      "timm                     0.9.7\n",
      "tokenizers               0.19.1\n",
      "torch                    2.0.1\n",
      "torchaudio               2.0.2\n",
      "torchvision              0.15.2\n",
      "tornado                  6.4.1\n",
      "tqdm                     4.62.3\n",
      "traitlets                5.14.3\n",
      "transformers             4.44.2\n",
      "triton                   2.0.0\n",
      "typing_extensions        4.12.2\n",
      "urllib3                  2.2.3\n",
      "wcwidth                  0.2.13\n",
      "wheel                    0.44.0\n",
      "widgetsnbextension       4.0.13\n",
      "xxhash                   3.5.0\n",
      "yarl                     1.16.0\n",
      "zipp                     3.20.2\n"
     ]
    }
   ],
   "source": [
    "!pip list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s9vsRI0elwem"
   },
   "source": [
    "## Config Method Parameters\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f2kC4WkAmhfV"
   },
   "source": [
    "```python\n",
    "# For IKE hparams:\n",
    "alg_name: \"IKE\"\n",
    "model_name: \"gpt2\" or \"llama-7B\"\n",
    "sentence_model_name: \"all-MiniLM-L6-v2\"\n",
    "device: 0\n",
    "results_dir: \"./results\"\n",
    "\n",
    "k: 32\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "Pp6wahHlBxHZ",
    "outputId": "6a27c3e2-2e6b-43cc-cf81-23d8b5ad8b27"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: matplotlib in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (3.5.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (4.54.1)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (1.4.7)\n",
      "Requirement already satisfied: numpy>=1.17 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (1.22.1)\n",
      "Requirement already satisfied: packaging>=20.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (24.1)\n",
      "Requirement already satisfied: pillow>=6.2.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (11.0.0)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (3.2.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from matplotlib) (2.9.0.post0)\n",
      "Requirement already satisfied: six>=1.5 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
      "Looking in indexes: https://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: sentence_transformers in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (3.2.1)\n",
      "Requirement already satisfied: transformers<5.0.0,>=4.41.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (4.44.2)\n",
      "Requirement already satisfied: tqdm in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (4.62.3)\n",
      "Requirement already satisfied: torch>=1.11.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (2.0.1)\n",
      "Requirement already satisfied: scikit-learn in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (1.0.2)\n",
      "Requirement already satisfied: scipy in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (1.7.3)\n",
      "Requirement already satisfied: huggingface-hub>=0.20.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (0.25.1)\n",
      "Requirement already satisfied: Pillow in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sentence_transformers) (11.0.0)\n",
      "Requirement already satisfied: filelock in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from huggingface-hub>=0.20.0->sentence_transformers) (3.16.1)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from huggingface-hub>=0.20.0->sentence_transformers) (2024.10.0)\n",
      "Requirement already satisfied: packaging>=20.9 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from huggingface-hub>=0.20.0->sentence_transformers) (24.1)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from huggingface-hub>=0.20.0->sentence_transformers) (6.0)\n",
      "Requirement already satisfied: requests in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from huggingface-hub>=0.20.0->sentence_transformers) (2.32.3)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from huggingface-hub>=0.20.0->sentence_transformers) (4.12.2)\n",
      "Requirement already satisfied: sympy in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (1.13.3)\n",
      "Requirement already satisfied: networkx in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (3.2.1)\n",
      "Requirement already satisfied: jinja2 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (3.1.4)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.7.99)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.7.99)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.7.101)\n",
      "Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (8.5.0.96)\n",
      "Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.10.3.66)\n",
      "Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (10.9.0.58)\n",
      "Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (10.2.10.91)\n",
      "Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.4.0.1)\n",
      "Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.7.4.91)\n",
      "Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (2.14.3)\n",
      "Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (11.7.91)\n",
      "Requirement already satisfied: triton==2.0.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from torch>=1.11.0->sentence_transformers) (2.0.0)\n",
      "Requirement already satisfied: setuptools in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.11.0->sentence_transformers) (75.1.0)\n",
      "Requirement already satisfied: wheel in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.11.0->sentence_transformers) (0.44.0)\n",
      "Requirement already satisfied: cmake in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from triton==2.0.0->torch>=1.11.0->sentence_transformers) (3.30.5)\n",
      "Requirement already satisfied: lit in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from triton==2.0.0->torch>=1.11.0->sentence_transformers) (18.1.8)\n",
      "Requirement already satisfied: numpy>=1.17 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from transformers<5.0.0,>=4.41.0->sentence_transformers) (1.22.1)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from transformers<5.0.0,>=4.41.0->sentence_transformers) (2024.9.11)\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from transformers<5.0.0,>=4.41.0->sentence_transformers) (0.4.5)\n",
      "Requirement already satisfied: tokenizers<0.20,>=0.19 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from transformers<5.0.0,>=4.41.0->sentence_transformers) (0.19.1)\n",
      "Requirement already satisfied: joblib>=0.11 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from scikit-learn->sentence_transformers) (1.4.2)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from scikit-learn->sentence_transformers) (3.5.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from jinja2->torch>=1.11.0->sentence_transformers) (3.0.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from requests->huggingface-hub>=0.20.0->sentence_transformers) (3.4.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from requests->huggingface-hub>=0.20.0->sentence_transformers) (3.10)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from requests->huggingface-hub>=0.20.0->sentence_transformers) (2.2.3)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from requests->huggingface-hub>=0.20.0->sentence_transformers) (2024.8.30)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages (from sympy->torch>=1.11.0->sentence_transformers) (1.3.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install matplotlib\n",
    "!pip install sentence_transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OMGZhZ7NmphY"
   },
   "source": [
    "## Import modules & Run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/mnt/8t/xkw/EasyEdit\n"
     ]
    }
   ],
   "source": [
    "%cd .."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For GPT Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "### For GPT Model\n",
    "from easyeditor import BaseEditor\n",
    "from easyeditor import IKEHyperParams\n",
    "from easyeditor.models.ike.util import encode_ike_facts\n",
    "from sentence_transformers import SentenceTransformer\n",
    "\n",
    "\n",
    "prompts = ['Q: The president of the US is? A:',]\n",
    "ground_truth = ['Donald Trump']\n",
    "target_new = ['Joe Biden']\n",
    "subject = ['president']\n",
    "rephrase_prompts = ['The leader of the United State is']\n",
    "\n",
    "# IKE need train_ds(For getting In-Context prompt)\n",
    "train_ds = [\n",
    "    {\n",
    "        'prompt': 'Q: The president of the US is? A:',\n",
    "        'target_new': 'Joe Biden',\n",
    "        'rephrase_prompt': 'The leader of the United State is',\n",
    "        'locality_prompt': 'The president of Russia is ',\n",
    "        'locality_ground_truth': 'Putin'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Einstein specialized in',\n",
    "        'target_new': 'physics',\n",
    "        'rephrase_prompt': 'Einstein is good at',\n",
    "        'locality_prompt': 'Q: Which subject did Newton specialize in? A: ',\n",
    "        'locality_ground_truth': 'physics'\n",
    "\n",
    "    },\n",
    "    # add more if needed\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-12-01 14:16:45,605 - easyeditor.editors.editor - INFO - Instantiating model\n",
      "12/01/2024 14:16:45 - INFO - easyeditor.editors.editor -   Instantiating model\n",
      "/mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884\n",
      "  warnings.warn(\n",
      "2024-12-01 14:16:47,134 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left...\n",
      "12/01/2024 14:16:47 - INFO - easyeditor.editors.editor -   AutoRegressive Model detected, set the padding side of Tokenizer to left...\n",
      "12/01/2024 14:16:48 - INFO - sentence_transformers.SentenceTransformer -   Use pytorch device_name: cuda\n",
      "12/01/2024 14:16:48 - INFO - sentence_transformers.SentenceTransformer -   Load pretrained SentenceTransformer: ./hugging_cache/all-MiniLM-L6-v2\n"
     ]
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.006377220153808594,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Batches",
       "rate": null,
       "total": 1,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a14c598a6bb943658e523288a6c8b06c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1/1 [00:00<00:00,  1.13it/s]\n",
      "  0%|          | 0/1 [00:00<?, ?it/s]12/01/2024 14:16:52 - INFO - sentence_transformers.SentenceTransformer -   Use pytorch device_name: cuda\n",
      "12/01/2024 14:16:52 - INFO - sentence_transformers.SentenceTransformer -   Load pretrained SentenceTransformer: ./hugging_cache/all-MiniLM-L6-v2\n",
      "/mnt/8t/xkw/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884\n",
      "  warnings.warn(\n",
      "100%|██████████| 1/1 [00:00<00:00,  5.72it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['New Fact: Q: The president of the US is? A: Joe Biden\\nPrompt: Q: The president of the US is? A: Joe Biden\\n\\n', 'New Fact: Q: The president of the US is? A: Joe Biden\\nPrompt: The leader of the United State is Joe Biden\\n\\n', 'New Fact: Q: The president of the US is? A: Joe Biden\\nPrompt: The president of Russia is  Putin\\n\\n', 'New Fact: Einstein specialized in physics\\nPrompt: Einstein is good at physics\\n\\n', 'New Fact: Einstein specialized in physics\\nPrompt: Einstein specialized in physics\\n\\n', 'New Fact: Einstein specialized in physics\\nPrompt: Q: Which subject did Newton specialize in? A:  physics\\n\\n', 'New Fact: Q: The president of the US is? A: Joe Biden\\nPrompt: Q: The president of the US is? A: Joe Biden\\n\\n']\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-12-01 14:16:53,162 - easyeditor.editors.editor - INFO - 0 editing: Q: The president of the US is? A: -> Joe Biden  \n",
      "\n",
      " {'pre': {'rewrite_acc': 0.5, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}\n",
      "12/01/2024 14:16:53 - INFO - easyeditor.editors.editor -   0 editing: Q: The president of the US is? A: -> Joe Biden  \n",
      "\n",
      " {'pre': {'rewrite_acc': 0.5, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Metrics Summary:  {'pre': {'rewrite_acc': 0.5, 'rephrase_acc': 1.0}, 'post': {'rewrite_acc': 1.0, 'rephrase_acc': 1.0}}\n",
      "[{'pre': {'rewrite_acc': 0.5, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}]\n"
     ]
    }
   ],
   "source": [
    "hparams = IKEHyperParams.from_hparams('./hparams/IKE/gpt2-xl')\n",
    "editor = BaseEditor.from_hparams(hparams)\n",
    "# Initialize SentenceTransformer model\n",
    "sentence_model = SentenceTransformer(hparams.sentence_model_name)\n",
    "# Generate and save sentence embeddings\n",
    "encode_ike_facts(sentence_model, train_ds, hparams)\n",
    "\n",
    "metrics, edited_model, _ = editor.edit(\n",
    "    prompts=prompts,\n",
    "    ground_truth=ground_truth,\n",
    "    rephrase_prompts=rephrase_prompts, # new para\n",
    "    target_new=target_new,\n",
    "    subject=subject,\n",
    "    train_ds=train_ds,\n",
    "    sequential_edit=True\n",
    ")\n",
    "\n",
    "print(metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.\n"
     ]
    }
   ],
   "source": [
    "\n",
    "from transformers import GPT2Tokenizer\n",
    "from transformers import GPT2LMHeadModel\n",
    "\n",
    "tokenizer = GPT2Tokenizer.from_pretrained('./hugging_cache/gpt2-xl',trust_remote_code=True)\n",
    "tokenizer.pad_token_id = tokenizer.eos_token_id\n",
    "tokenizer.padding_side='left'\n",
    "device = 1\n",
    "model = GPT2LMHeadModel.from_pretrained('./hugging_cache/gpt2-xl',trust_remote_code=True).to(f'cuda:{device}')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************\n",
      "Pre-Edit  Output:  It's blue.\n",
      "Post-Edit Output:  Red.  \n",
      "----------------------------------------------------------------------------------------------------\n",
      "Pre-Edit  Output:  Barack Obama. Q\n",
      "Post-Edit Output:  Biden.  \n",
      "----------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "ike_generation_prompts = [\n",
    "    \"The sky is red. \\\n",
    "    The color of the sky is red. \\\n",
    "    Q: What is the color of sky? A: Red. \\\n",
    "    Q: What color is the sky? A:\",\n",
    "    \"The president of the US is Biden. \\\n",
    "    Q: Who is the president of the US? A: Biden. \\\n",
    "    Biden is the leader of the United State. \\\n",
    "    Q: Who is the president of the US? A:\",\n",
    "]\n",
    "generation_prompts = [\n",
    "    \"Q: What color is the sky? A:\",\n",
    "    \"Q: Who is the president of the US? A:\",\n",
    "]\n",
    "\n",
    "batch = tokenizer(generation_prompts, return_tensors='pt', padding=True)\n",
    "edited_batch = tokenizer(ike_generation_prompts, return_tensors='pt', padding=True)\n",
    "\n",
    "pre_edit_outputs = model.generate(\n",
    "    input_ids=batch['input_ids'].to(model.device),\n",
    "    attention_mask=batch['attention_mask'].to(model.device),\n",
    "    max_new_tokens=4,\n",
    "    pad_token_id = tokenizer.eos_token_id\n",
    ")\n",
    "post_edit_outputs = edited_model.generate(\n",
    "    input_ids=edited_batch['input_ids'].to(edited_model.device),\n",
    "    attention_mask=edited_batch['attention_mask'].to(edited_model.device),\n",
    "    max_new_tokens=4,\n",
    "    pad_token_id = tokenizer.eos_token_id\n",
    ")\n",
    "print('*'*100)\n",
    "\n",
    "generation_max_length = batch['input_ids'].shape[-1]\n",
    "edited_max_length = edited_batch['input_ids'].shape[-1]\n",
    "for i in range(len(ike_generation_prompts)):\n",
    "    print(f'Pre-Edit  Output: {tokenizer.decode( pre_edit_outputs[i][generation_max_length :], skip_special_tokens=True)}')\n",
    "    print(f'Post-Edit Output: {tokenizer.decode(post_edit_outputs[i][edited_max_length :], skip_special_tokens=True)}')\n",
    "    print('--'*50 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For LlaMA Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GhbCFm6PkNRu"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/mnt/tbozhong/anaconda3/envs/serac/lib/python3.8/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'case_id': 0, 'time': 0.25104260444641113, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}, 'pre': {'rewrite_acc': 0.5, 'portability': {}, 'rephrase_acc': 1.0}}]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "normalizer.cc(51) LOG(INFO) precompiled_charsmap is empty. use identity normalization.\n",
      "Loading checkpoint shards: 100%|██████████| 33/33 [00:11<00:00,  2.80it/s]\n",
      "/mnt/tbozhong/anaconda3/envs/serac/lib/python3.8/site-packages/transformers/tokenization_utils_base.py:2372: UserWarning: `max_length` is ignored when `padding`=`True` and there is no truncation strategy. To pad to max length, use `padding='max_length'`.\n",
      "  warnings.warn(\n",
      "/mnt/tbozhong/anaconda3/envs/serac/lib/python3.8/site-packages/transformers/generation/utils.py:1219: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)\n",
      "  warnings.warn(\n",
      "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n",
      "Input length of input_ids is 50, but `max_length` is set to 30. This can lead to unexpected behavior. You should consider increasing `max_new_tokens`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************\n",
      "Pre-Edit Outputs:  ['<unk><unk><unk>Q: What color is the sky? A: Blue. Q: What color is the grass? A: Green. Q: What', '<unk>Q: Who is the president of the US? A: Barack Obama B: George Bush C: Bill Clinton D: John F']\n",
      "Post-Edit Outputs:  ['<unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk>The sky is red.     The color of the sky is red.     Q: What is the color of sky? A: Red.     Q: What color is the sky? A: Red', '<unk>The president of the US is Biden.     Q: Who is the president of the US? A: Biden.     Biden is the leader of the United State.     Q: Who is the president of the US? A: B']\n"
     ]
    }
   ],
   "source": [
    "from easyeditor import BaseEditor\n",
    "from easyeditor import IKEHyperParams\n",
    "from easyeditor.models.ike.util import encode_ike_facts\n",
    "from sentence_transformers import SentenceTransformer\n",
    "\n",
    "\n",
    "prompts = ['Q: The president of the US is? A:',]\n",
    "ground_truth = ['Donald Trump']\n",
    "target_new = ['Joe Biden']\n",
    "subject = ['president']\n",
    "rephrase_prompts = ['The leader of the United State is']\n",
    "\n",
    "# IKE need train_ds(For getting In-Context prompt)\n",
    "train_ds = [\n",
    "    {\n",
    "        'prompt': 'Q: The president of the US is? A:',\n",
    "        'target_new': 'Joe Biden',\n",
    "        'rephrase_prompt': 'The leader of the United State is',\n",
    "        'locality_prompt': 'The president of Russia is ',\n",
    "        'locality_ground_truth': 'Putin'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Einstein specialized in',\n",
    "        'target_new': 'physics',\n",
    "        'rephrase_prompt': 'Einstein is good at',\n",
    "        'locality_prompt': 'Q: Which subject did Newton specialize in? A: ',\n",
    "        'locality_ground_truth': 'physics'\n",
    "\n",
    "    },\n",
    "    # add more if needed\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hparams = IKEHyperParams.from_hparams('./hparams/IKE/llama-7b')\n",
    "editor = BaseEditor.from_hparams(hparams)\n",
    "# Initialize SentenceTransformer model\n",
    "sentence_model = SentenceTransformer(hparams.sentence_model_name)\n",
    "# Generate and save sentence embeddings\n",
    "encode_ike_facts(sentence_model, train_ds, hparams)\n",
    "\n",
    "metrics, edited_model, _ = editor.edit(\n",
    "    prompts=prompts,\n",
    "    ground_truth=ground_truth,\n",
    "    rephrase_prompts=rephrase_prompts, # new para\n",
    "    target_new=target_new,\n",
    "    subject=subject,\n",
    "    train_ds=train_ds,\n",
    "    sequential_edit=True    \n",
    ")\n",
    "print(metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import LlamaForCausalLM\n",
    "from transformers import LlamaTokenizer\n",
    "\n",
    "device = 1\n",
    "model = LlamaForCausalLM.from_pretrained('hugging_cache/llama-2-7b-chat').to(f'cuda:{device}')\n",
    "tokenizer = LlamaTokenizer.from_pretrained('hugging_cache/llama-2-7b-chat')\n",
    "tokenizer.pad_token_id = tokenizer.eos_token_id\n",
    "tokenizer.padding_side = 'left'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "ike_generation_prompts = [\n",
    "    \"The sky is red. \\\n",
    "    The color of the sky is red. \\\n",
    "    Q: What is the color of sky? A: Red. \\\n",
    "    Q: What color is the sky? A:\",\n",
    "    \"The president of the US is Biden. \\\n",
    "    Q: Who is the president of the US? A: Biden. \\\n",
    "    Biden is the leader of the United State. \\\n",
    "    Q: Who is the president of the US? A:\",\n",
    "]\n",
    "generation_prompts = [\n",
    "    \"Q: What color is the sky? A:\",\n",
    "    \"Q: Who is the president of the US? A:\",\n",
    "]\n",
    "\n",
    "batch = tokenizer(generation_prompts, return_tensors='pt', padding=True)\n",
    "edited_batch = tokenizer(ike_generation_prompts, return_tensors='pt', padding=True)\n",
    "\n",
    "pre_edit_outputs = model.generate(\n",
    "    input_ids=batch['input_ids'].to(model.device),\n",
    "    attention_mask=batch['attention_mask'].to(model.device),\n",
    "    max_new_tokens=4,\n",
    "    pad_token_id = tokenizer.eos_token_id\n",
    ")\n",
    "post_edit_outputs = edited_model.generate(\n",
    "    input_ids=edited_batch['input_ids'].to(edited_model.device),\n",
    "    attention_mask=edited_batch['attention_mask'].to(edited_model.device),\n",
    "    max_new_tokens=4,\n",
    "    pad_token_id = tokenizer.eos_token_id\n",
    ")\n",
    "print('*'*100)\n",
    "\n",
    "generation_max_length = batch['input_ids'].shape[-1]\n",
    "edited_max_length = edited_batch['input_ids'].shape[-1]\n",
    "for i in range(len(ike_generation_prompts)):\n",
    "    print(f'Pre-Edit  Output: {tokenizer.decode( pre_edit_outputs[i][generation_max_length :], skip_special_tokens=True)}')\n",
    "    print(f'Post-Edit Output: {tokenizer.decode(post_edit_outputs[i][edited_max_length :], skip_special_tokens=True)}')\n",
    "    print('--'*50 )\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "EasyEdit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.20"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
